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ABSTRACT: Smart construction involves a growing array of devices that generate extensive data, capable of 
enhancing construction efficiency and productivity. Nonetheless, the handling of this diverse and abundant 
information, along with the geographical spread of construction sites, poses challenges to effective communication 
and information processing within the management system. Multi-robot systems, as a new type of Internet of Things 
device, have the potential ability to coordinate workers to complete their work while serving as an edge node for 
information storage and processing. This paper presents a multi-robot federated edge learning framework that 
facilitates construction information management and communication. The work demonstrates the role of 
distributed databases in processing information during project execution, in contrast to centralized information 
systems. To address the intricacies of construction sites and the wide array of equipment involved, unmanned 
aerial vehicles and quadruped robots are employed as edge nodes. The formation of a federated edge learning 
framework ensures the real-time processing of massive data and data privacy issues. The Federated Multi-Robot 
(FedMR) framework is a global sharing model focused on preserving differential privacy protection. This 
framework is distributed to multiple edge robots in each round, enabling local real-time processing of robot tasks. 
The system can accomplish target detection and tracking of workers based on computer vision. Additionally, we 
collect MiC energy consumption data during the construction process and predict carbon emissions. Based on the 
implementation and testing of the system, it has been shown to provide structured and reliable information, fast 
local transmission, and the ability to process information in real-time. The system's ability to coordinate workers 
and process information makes it a valuable tool in smart construction. 
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1. INTRODUCTION 


The increasing development of the construction industry towards being smart and the concern about management 
informatics stimulate a higher requirement for adopting construction technology. Internet of Things (IoT), 
blockchain smart contracts, and AI in construction are emerging as the next wave in smart construction, with 
examples such as detecting the presence of objects in the construction environment to improve safety (Fang et al., 
2018). The management of complex projects will be efficient, automated, and intelligent with computer vision 
technology (Xu et al., 2021). Prefabrication involves the use of different components from different manufacturers. 
This makes it difficult to standardize data sharing across the industry, leading to a lack of consistency in the data 
that is shared. And Prefabricated buildings, spearheaded by modular integrated construction (MiC) as a future 
trend in the construction industry, can lead to poor data sharing and communication resulting in the uniqueness of 
their supply chain (Wuni et al., 2022). The construction process is different from traditional construction methods. 
Prefabricated buildings are constructed off-site in a factory-controlled environment, where the materials and labor 
are streamlined and optimized for efficiency. This requires a unique supply chain that is focused on the 
procurement and delivery of materials to the factory, as well as the transportation and installation of the finished 
product to the construction site. Construction usually requires the cooperation of many stakeholders, which can be 
divided into four categories according to their functions: client, manufacturer, logistics company, and contractor. 
The process of modular construction involves various aspects of design, engineering, manufacturing, logistics, 
installation, and project management. Each stakeholder brings a unique set of skills and expertise to the project, 
including architects, engineers, contractors, manufacturers, transportation specialists, and project managers. And 
these multidisciplinary stakeholders have different expectations, interests, and motivations, and the plethora of 
participants in a construction project leads to low information transparency, inefficient transactions, and even 
frauds. (Luo et al., 2019) Today’s construction is operating in highly dynamic environments, which requires the 
information facilities to be able to provide stable network services and adequate computing sources in interaction 
with the environment on site. For example, verifying the statuses of the modules can monitor the construction 
progress . At the same time, the high level of privacy and the large amount of information generated during the 
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project management process leads to a decrease in efficiency. In order to effectively tackle the challenges posed 
by inadequate infrastructure and privacy concerns during the construction process, it is imperative to implement a 
robust on-site system architecture that facilitates mobile crowdsensing, shared storage, and processing of 
information from computational sources while ensuring data privacy. This technology is critical for the 
advancement of Construction 4.0, and can only be accomplished with ample computing resources and a reliable 
network. By bringing information technology to this traditional yet modern field, we can revolutionize the 
construction industry. The system can improve information transparency, reduce information asymmetry and 
facilitate collaborative work throughout the construction project lifecycle. Managers can keep track of the project's 
construction progress in real-time, identify problems and solve them in time to avoid causing construction 
problems such as deliveries delay, the absence of workers, machinery breakdowns, etc. Effective information 
sharing can also coordinate tasks between managers and workers, improve communication efficiency and optimize 
the construction process (Jiang et al., 2021). 


The limitations of information technology, such as data transmission, make communication and data visualization 
in construction less efficient (Niu et al., 2015). This has a significant impact on the design of low-energy buildings, 
which require real-time monitoring of the environment and control. Niu et al. proposed A virtual reality integrated 
design approach to improving occupancy information integrity for closing the building energy performance gap 
(Niu et al., 2016). But this approach also generates a lot of private data due to the fact that the attitude of companies 
such as manufacturers towards new technologies depends on the environmental and organizational context (Pan 
& Pan, 2019). Building organizations can have concerns about the use of technologies that contain similar issues. 
In turn, this data cannot be fully utilized in a shared manner. A framework that can break through the efficiency of 
data transfer and address data privacy is necessary for the construction process. 
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Fig. 1. An example of construction information management by federated learning 


Fig. 1 Illustrates the multi-robot horizontal federated edge learning framework for construction information 
Management. The framework enables robots to gather information from workers and vehicles and transmit it in 
real-time through sensors to the robots. The robots then sample the information and process their query on the 
device. For instance, if one frame includes a vehicle, the robot will detect and send back the data to the user while 
another section of the sampled frame will be forwarded to the edge server for object detection, specifically the 
detection of the vehicle. Throughout the video analysis query processing lifecycle, the edge device and edge server 
collaborate to provide the user with detection results. By utilizing this framework, the system can effectively 
manage construction information by collecting and analyzing data from various sources, ultimately improving 
project efficiency, and reducing errors. 
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The increasing development of smart construction and the concern about the application of the IoT in construction 
stimulate a higher requirement for information exchange and communication quality. There are privacy issues in 
the transmission, storage, and analysis of data. These data relate to confidential industry information, such as the 
operations and costs of the companies involved. Institutional restrictions hinder information sharing between 
clients and contractors. Strict regulations undermine the efficiency of information sharing between clients and 
contractors. As factories, shipping companies, and contractors are independent. They cannot share data between 
them, which makes it difficult for project management as a third-party company to organize and coordinate the 
entire construction process. Site coordination requires the exchange of information between different contractors 
and sharing raw data between different contractors may compromise privacy and lead to problems such as 
communication barriers. The lack of information sharing between construction companies and suppliers is a 
significant barrier (Ojo et al., 2014) . Smart Construction is a method of construction that requires coordination in 
construction duration. A highly coordinated construction program of plant, transport, and construction is required. 
However, if clients and contractors do not accept relational matters as a long-term strategy, they will refuse to 
share implicit information (Yan, 2014). 


Insufficient computing power equipment on site due to the complex environment of the construction site and the 
lack of well-established power and communication system. The raw data collected on the edge devices need to be 
uploaded to a cloud server for processing, which requires a lot of data transfer and processing time. For the 
construction of security systems, functions such as path planning and target recognition require real-time 
computing capabilities. While the majority of devices currently possess at least one image sensor and the ability 
to record and play high-resolution videos, they are deficient in the processing power required to execute intricate 
real-time computer vision algorithms. They are still unable to perform high-intensity performance computer vision 
tasks in real-time, with high frame rates and low latency (Honegger et al., 2014). 


Star topology is widely used when the intelligence of the network is concentrated on the central node. However, 
the star topology has many disadvantages (Bisht & Singh, 2015). The star topology has many redundant links to 
ensure high connectivity, which results in high installation and maintenance costs and poor resource-sharing 
capabilities. Also, the communication lines are only used by the central and edge nodes on the line, which lead to 
communication lines being poorly utilized. Nevertheless, the central node demands frequent attention, and if it 
malfunctions, the entire network will come to a standstill. As computing evolves from centralized mainframe 
systems to many powerful microcomputers and workstations, the use of the traditional star topology will be 
reduced. 


The purpose of this study is to introduce a multi-robot federated edge learning framework that can facilitate 
communication and information management in smart construction. The paper highlights the challenges posed by 
the large amount and wide variation of information involved in construction and the spatial dispersion of 
construction sites to the information management system (Akinosho et al., 2020). The paper propose the use of 
unmanned aerial vehicles and quadruped robots as edge nodes to process information during project execution. 
The framework is a worldwide sharing model founded on the principles of differential privacy protection. It gets 
distributed to numerous edge robots during each round, enabling local real-time processing of robot tasks. The 
system can accomplish target detection and tracking of workers based on computer vision, and predict carbon 
emissions. The authors demonstrate that the system can provide structured and reliable information, fast local 
transmission, and the ability to process information in real-time, making it a valuable tool in smart construction. 


This section below sets the scene by giving an overview of the existing initiatives and status concerning 
construction information systems. The rest of the paper is structured as follows: Section 2 reviews the literature of 
federated learning, federated learning in construction, and distributed ledger technology. Section 3 discusses the 
research methodology and object selection. Section 4 presents the system architectures and federated edge learning. 
Section 5 discusses the applications to the construction industry and performance evaluation of the federated edge 
learning framework. Finally, Section 6 draws the conclusions and suggests future research. 


2. LITERATURE REVIEW 
2.1 Federated learning 


Federated learning (FL) is a distributed machine learning scheme based on parallel computing that can overcome 
the challenges of data sensibility and data silos through the collaborative and decentralized neural network 
structure. FL has a high correlation with distributed learning, while it focuses on providing a collaborative model 
without privacy leakage (Li et al., 2020) At present, the FL has two mainstream open-source frameworks. Google 
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(Google, 2019) proposed a TensorFlow Federated (TFF) framework for meeting the demands of deep learning 
services in decentralized data. Webank (“Webank (2019a),” n.d.) presents the first industrial-level framework, 
Federated AI Technology Enabler (FATE) serves for cross-organizational architecture. Furthermore, Ramaswamy 
et al. (Ramaswamy et al., 2019) proposed the prediction of emoji in mobile keyboards, which is a successful 
application for model improvement and Secure Aggregation for the concern of stakeholder privacy. And Yang et 
al. (Yang et al., 2019) divide FL frameworks into three categories: vertical FL, horizontal FL, and federated transfer 
learning. In the case of vertical Fl, data is partitioned in the vertical direction by the feature dimension. Horizontal 
FL is suitable for cases in which data are multiple in sample space and a set number of overlaps among the feature 
of data storage in various nodes. Upon most occasions, data shares are quite distinct from sample space and feature 
space. Therefore, federated transfer learning can solve the problem in this setting is poor data quality without data 
labels (Li et al., 2020). 


2.2 Construction information management 


Having access to data at the right time for construction managers can assist project construction in assessing the 
construction performance of the corporation and subcontractors (Carrillo et al., 2013). When implementing 
construction information management, accurate data recording and comprehensive data analysis help to establish 
the credibility of stakeholders (Yang et al., 2019). Construction is a complex, one-off manufacturing process that 
involves many businesses, including design, manufacture, and transport. To assess the risks linked to each 
stakeholder, unprocessed data is necessary. Forecasting stems from data mining, a result of the methodical 
management of construction data. In addition, based on the data simulations, the manager can predict the potential 
problem area and plan for them (Arayici et al., 2012). With the completion of the first phase, the requirement for 
additional equipment and material in the second phase is identified. The analysis also provides construction 
stakeholders with visual information on project progress (Doloi, 2013). With the better access to and updating of 
the management system, the raw data should not be recorded with simple storage. It also provides data analysis 
servers. The potential for innovative algorithms based on artificial intelligence to support efficient construction 
processes (Pan et al., 2022). Data analytics and IoT are useful for the analysis of construction impact on parameters. 
In construction, construction information management is as important as construction, as information affects all 
construction-related activities (Kim et al., 2013). 


3. METHODOLOGY 
3.1 Problem definition 


The primary focus of this present research is on information processing, analysis, and collation in construction 
scenarios. To achieve the study aim, construction robots have been designed as computational nodes, each 
corresponding to a construction safety monitoring device. These monitoring devices include cameras such as RGB, 
RGB-D, and LIDAR. Construction robots can be integrated as computational nodes, and can assist in the 
construction process while also collecting data and training models locally. This local processing ensures that data 
safety functions can be achieved while analyzing data on-site. In the context of construction safety monitoring, 
construction robots equipped with safety monitoring devices can help ensure safety compliance in construction 
sites. Additionally, they can help identify potential safety hazards, such as structural instability or unsafe working 
conditions. Furthermore, the collected data can be processed and analyzed locally, without the need to transfer 
sensitive information to external servers, ensuring data safety and privacy. By doing so, data can be analyzed in 
real-time, enabling quick decisions and actions to be taken to prevent accidents or hazards. 


By utilizing the FedMR approach, data sharing is minimized, and data privacy is protected while effectively 
predicting the facial fatigue status of workers in real-time. This approach can aid in improving construction safety 
by enabling prompt interventions, when necessary, ultimately reducing accidents and improving the overall safety 
of workers and equipment vehicles in the construction scene. 


3.2 Construction information framework overview 


Construction is a complex process that generates a vast amount of information, as depicted in Fig. 2. The 
construction information is created from the design phase at the start of the project, and all project-related data is 
stored in a distributed manner on the construction site. This includes the management of approvals and material 
information, installation and transportation processes, and the information can be collaboratively managed. 
Through federated learning, all resources can be integrated securely, and machine learning can be used to 
efficiently integrate resources and make plans. Construction projects are complex endeavors that involve a 
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SECTION B - ADVANCED PROJECT MANAGEMENT AND CONTROL 


multitude of stakeholders, including architects, engineers, contractors, and subcontractors. 
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Fig. 2. Construction information on the different phase 


Coordinating the information flow among these entities can be challenging, especially given the extensive data 
that requires sharing and processing. Federated learning presents a promising remedy, providing a secure and 
efficient approach for construction information management. It enhances efficiency and accuracy without 
compromising data privacy and security, as multiple parties collaboratively train a model without sharing raw data. 
When well-implemented, federated learning aids stakeholders in early problem identification and informed 
decision-making, ensuring project success. 


Edge Controller 


Optimizer + Approximator 


Control process Control process 


O 
i | ] fp 
7 
Workflow Workflow Pig 
H- BO re 
- - -p Se > - = = = 


Quadruped robot/UAV Edge Server 
— ----> 
Control path Data path 


Fig. 3. FedMR system framework overview 


The Federated Multi-Robot (FedMR) framework consists of an edge controller, a series of multi-robot systems, 
and servers, as illustrated in Fig. 3. The data flow involves two routes: the data path and the control path. A user's 
query traverses through the robots and servers in the workflow, which consists of a series of functions. In the data 
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path, construction takes place sequentially from the first function to the last one in the workflow. Following data 
processing through this workflow, the resulting outputs are then delivered back to the user. 


By employing the FedMR framework, construction information can be managed efficiently and securely, enabling 
prompt interventions when necessary and improving the overall safety of workers and equipment vehicles on the 
construction site. 


4. APPLICABILITY OF THE FEDMR SYSTEM 


To validate the viability of the approach, experiments were conducted to detect targets in images using a quadruped 
robot that captured local video data. The system carried out target recognition at the robot's side when the edge 
server retrieved images of vehicles. The training dataset for detecting moving objects in construction sites (MOCS) 
was preloaded into the robot. 


In essence, the system's target detection capabilities were put to the test using real-world scenarios. By setting up 
video data collected by the quadruped robot, the system was able to accurately identify and classify targets within 
the images. The edge server, which received images of vehicles, promptly recognized the target objects, thanks to 
the training dataset preloaded in the robot. The MOCS dataset proved invaluable in training the system to detect 
moving objects in construction sites. By integrating the dataset into the robot, the system was able to recognize 
and classify objects in real-time, even in complex construction environments. This approach demonstrates the 
effectiveness of preloading training datasets into robots, enabling them to perform target recognition efficiently 
and accurately. 


4.1 Environment setting for image test 


The experiment, depicted in Fig. 4, employs YOLOv8 on Pytorch 1.8.1. The NVIDIA RTX3090 GPU serves as 
the platform for training and testing the model. Due to the rough road surfaces present in the construction site, a 
quadrupedal robot was selected as the ideal load-carrying system, as a typical wheeled chassis would have 
struggled to navigate the build environment with ease.The industrial camera used in the experiment is DF100- 
1080P (JIERUIWEITONG), while Unitree Gol and DJI M200 function as edge devices. During the training phase, 
the new model incorporates parts of the pre-trained model from YOLOv8x. Since YOLOv8 and YOLOv8x share 
most of the backbone (block 0*8) and some of the head, it is possible to transfer a wide range of weights from 
YOLOv8x. Leveraging these weights during training can save significant time and computational resources. 
Overall, the use of YOLOv8 on Pytorch 1.8.1, combined with the powerful NVIDIA RTX3090 GPU, enables the 
model to train and test efficiently. Additionally, the use of industrial cameras, as well as the Unitree Gol and DJI 
M200 edge devices, further enhances the experiment's reliability and accuracy. By incorporating pre-trained 
models and weights, the experiment provides a practical approach to target detection that can be easily adapted to 
a wide range of scenarios. 


Unitree ee 


Fig. 4. Quadruped robot-based edge devices 


558 


SECTION B - ADVANCED PROJECT MANAGEMENT AND CONTROL 


4.2 Object detection in federated edge learning 
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Fig. 6. Validated datasets with labels. 


As illustrated in Fig. 5 and Fig. 6, the model trained on the local data training set achieved impressive results in 
target recognition. All targets were recognized compared to the labelled images. However, some of the worker 
targets had smaller IoU values when they were smaller in size. On the other hand, large vehicles were well 
recognized if they were presented as a whole. The best results were obtained for excavators. 


When the edge server makes a query for a relevant target, the query image can be transferred back to the 
corresponding server. This enables the data to be trained and queried under local conditions, facilitating the 
management of information for construction while maintaining data security. By utilizing this method, 
administrators can access the information they need for security, management, etc., without having to access data 
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from other departments. 
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Fig. 7. The result of Fl-scores 


As shown in Fig. 7, the F1(H-mean) score is a combined evaluation indicator of recall and accuracy. The F1 value 
represents the division of the arithmetic mean by the geometric mean, with a higher value indicating better 
performance. Analyzing the Precision and Recall formulas in the context of this reveals that when the F1 value is 
low, there's a relative increase in True Positives and a decrease in False Positives, leading to a relative increase in 
both Precision and Recall. Essentially, F1 is a weighted measure that considers both Precision and Recall. 

2 2*P*R 
1,1 P+R 


The P is the precision of the YOLOv8 based on FedMR , and R is the recall of algorithm. 


The worker and crane identification had the best F1 score. The confidence of all categories was 0.72 at F1=0.357, 
which is a significant improvement compared to the YOLOv8 standard of 0.65. This indicates that confidence is 
also guaranteed to be very good at higher recall. This data proves that edge nodes can provide good data processing 
capabilities, and the local model can process the data collected in the field in real-time. With a confidence level of 
0.35 at 0.70 in comparison to YOLOV8 utilizing a non-FedMR framework, it is evident that the FedMR framework 
significantly enhances information processing capabilities. This improvement leads to better performance, even 
when data processing is restricted to a localized environment. 


The FedMR have adapted the original FederatedAveraging (FedAvg) algorithm to our framework as shown in 
Algorithm 1. This aim is to investigate the impact of various data divisions and federated learning settings. To 
achieve this, the framework have modified the FedAvg algorithm to a FedMR algorithm by replacing the server- 
client communication framework, such as SocketIO, with a method that saves and restores checkpoints on hard- 
devices. This simplifies the model aggregation process. However, our implementation can also be effortlessly 
transferred to FedAvg. 


Algorithm | FedMR 


Input: N client parties{C,,};,—1 y,total rounds T , and Server side S; 
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Output: Aggregated Model w 


S initializes federated model parameters, and saves as checkpoint. Client parties {C,},-1.y, load the 
checkpoints. 


for t = 1, ..„T do 
for k= 1, ..,T do 
wp = w 
each client {Cp} do local training: 
fori = 0,1,...,M@do 
(Mk is the number of data batches b in the client Cp) 
client {Ck} computes gradients VP(w , bi) 
update with wg = wy — nVE(Wx, bi) 
end for 
save W, results to checkpoints. 
end for 
S loads checkpoints and get averaged model with w = =YN_, wy 
end for 


Return w 


5. CONCLUSIONS 


FedMR has demonstrated its versatility as a comprehensive model for handling information, optimizing processes, 
and monitoring isolated data. This empowers robots to offer insights, predictions, and warnings tailored for 
individual distributed construction workers prior to task execution at the work package level. However, existing 
machine learning approaches for modeling, optimization, and monitoring in construction information management 
necessitate data sharing or aggregation from each company, posing privacy risks and failing to deliver personalized 
monitoring of construction site conditions. 


Therefore, this paper has proposed a horizontal federal learning framework, FedMR, to aggregate cryptographic 
information data parameters from different building stakeholders without compromising privacy and personalize 
the model differently based on the jobs each robot is responsible for. The model is improved based on the Fedvision 
model, utilizing the multi-robot as the edge device. The testing process is experimentally validated through the 
retrieval of image information by managers, using a target detection approach by training and testing locally. After 
experimental verification, the efficiency of YOLO algorithm can be improved in FedMR framework can be 
improved by 1.5% under the premise of user privacy F1 can be improved. The evaluation results show that the 
proposed FedMR can achieve mainstream YOLO recognition performance, and privacy is well protected. However, 
there is still room for improvement in terms of model performance and scalability. Future research should focus 
on exploring the potential of incorporating more advanced machine learning techniques such as the Large language 
model. Additionally, as the use of robots in construction becomes more prevalent, it is important to continue to 
prioritize the development of secure and personalized monitoring solutions to ensure the safety and efficiency of 
construction processes. 
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